Papers with textual descriptors
PEEB: Part-based Image Classifiers with an Explainable and Editable Language Bottleneck (2024.findings-naacl)
Copied to clipboard
| Challenge: | CLIP-based classifiers rely on the prompt containing a class name that is known to the text encoder and perform poorly on new classes or the classes whose names rarely appear on the Internet. |
| Approach: | They propose to use a set of text descriptors to express a class name into a textual descriptable and match the embeddings of the detected parts to their textual ones to compute a logit score. |
| Outcome: | The proposed classifier outperforms CLIP-based classifiers on zero-shot and supervised learning settings by 88.80% and 92.20% accuracy on CUB-200 and Stanford Dogs-120. |